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^ , Abstract 

We study image compression by a separable wavelet basis |'0(2'^i3; — i)ijj{2^'^y — 

j), (j){x - i)^{2^^y - j), ^l){2^^{x - i)(j){y - j), 4){x - i)(i){y - i)], where ki,k2 G 

^- ' Tjj^; i,j € Z; and (/>,'!/' are elements of a standard biorthogonal wavelet basis in 

^ . L2{M.). Because ki ^ k^, the supports of the basis elements are rectangles, and 

t/3 \ the corresponding transform is known as the rectangular wavelet transform. We 

^, ' prove that if one-dimensional wavelet basis has M dual vanishing moments then 

the rate of approximation by N coefficients of rectangular wavelet transform is 

„^ , 0{N~ log N) for functions with mixed derivative of order M in each direction. 

OO , The square wavelet transform yields the approximation rate is 0{N~^^''^) for 

^^ \ functions with all derivatives of the total order M. Thus, the rectangular wavelet 

\^ • transform can outperform the square one if an image has a mixed derivative. We 

^^ . provide experimental comparison of image compression which shows that rectangu- 



o 

O 



X 



lar wavelet transform outperform the square one. 

Key words: Non-linear wavelet compression, rectangular wavelet transform, 
hyperbolic wavelets, anisotropic Besov space, sparse grid. 



1 Introduction 



Wavelet analysis has become a powerful tool for image and signal processing. 
Initially developed for approximations and analysis of functions of a single 
real variable, wavelet techniques was almost immediately generalized for the 
case of two and many variables. 

We will start our discussion with the most recent generalization, non-separable 
wavelets. In this construction, lattice of integers in one dimensional case is 
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replaced by the quincunx or a general lattice in the multi-dimensional case, 
and a wavelet analysis is derived directly for this lattice (see [?,?])• 

Although the non-separable approach now attracts the majority of researchers, 
we will focus this paper on comparison of the square and the rectangular sepa- 
rable wavelets for the two-dimensional case. We will show that the rectangular 
wavelet transform in some applications is more suitable than classical square 
one. The "rectangularization" technique can also be applicable to some non- 
separable wavelet constructions. 

Separable approaches involve building a basis in L2(M'^) using elements of a 
wavelet basis for L2(M). DeVore et al. in [?] and numerous other researchers 
studied the following basis in L2(M^): 

{0(2'x - ^)0(2't/ - j), ^(2'^a; - ^)^(2^ - j), 

0(2'^a; - i)^^{2'y - j),^{2''x - t)<j>{2'y - j)}, (1) 

where is a scaling function, ip is the corresponding wavelet, I & Z, k > I, 
A; G Z, and i,j G Z. 

Because the supports of the basis functions are localized in the squares with 
sizes 0{2~^ X 2~^), the decomposition on this basis is called the square wavelet 
transform. It is well known that if a function has bounded (in a certain space) 
derivatives of the total order M and ip has M vanishing moments then the 
rate of approximation by N elements of this basis is 0{N~^^^). 




(b) 



Fig. 1. The Lena images decomposed by square and rectangular wavelet transforms 
based on the Haar basis. 

Rectangular wavelet transform is inspired by approximation theory technique 
known as "sparse grid", or "hyperbolic cross" (see, for example, [?]). The 
square wavelet transform (and it's higher dimensional analogs) suffers from so 



called "curse of dimensionality" . In order to get the same precision of approx- 
imation as by A^ coefficients in the one dimensional case, one needs to utilize 
0{N'^) coefficients for the function of d variables. The "sparse grid" technique 
allows to build a basis for functions of many variables that yields the same 
rate of convergence (up to a logarithmic factor) as in the case of one variable. 
In application to the construction of a wavelet basis, it yields a basis 

{V'(2'^^x - i)^lj{2^^y - j), 0(2'x - i)ij{2'''y - j), 

^{2''^{x - i)<P{2'y - J, (f){x - t)(j){y -?))}, (2) 

where ki, k2 > I, fci, ^2 G Z, i, j G Z and / is a fixed integer number. Because 
elements of the basis are supported on rectangles of size 0{2~''^ x 2~'^^), and 
ki 7^/1:2, the corresponding transform can be called rectangular. 

The application of this basis to the problem of multivariate surface denoising 
was studied in [?] and [?]. The approximation properties of the rectangular 
wavelet transform in Besov spaces was studied DeVore et al. in [?], and by 
Zavadsky in [?]. 

This paper aims at providing a self contained study of the approximation 
properties of the rectangular wavelet transform from the perspective of image 
processing and benchmarking of the rectangular transforms with respect to 
the square one for image compression. 

We show that if the approximated function has the mixed derivative of total 
order 2M and ip has M vanishing moments, then the rate of approximation 
by A^ elements of this basis is 0{N-^ log*^ A^). 

The rest of the paper is organized as follows. In section 2, we recall some basic 
properties of one dimensional wavelet bases. In section 3, we consider our 
motivations for migration from square wavelet transform to rectangular one. 
In section 4, we consider approximation rates of the non-linear approximations 
by the rectangular wavelet basis in Lp(R^), 1 < p < 00. In section 5, we provide 
numerical experiments results on standard test images. 



2 One-dimensional wavelet transform 



Biorthogonal bases of compactly supported wavelets were build in [?] . Follow- 
ing this paper, we assume that we have four compactly supported functions 
0, (p,ip,ip G Loo(M) and coefficients vectors h, h, g, g that satisfy the following 



conditions: 

(t){x)=Y.hi(t){2x-i), (3) 

0(a;)=^M(2a;-2), (4) 

^{x)=Y,9i<P{'^x-i), (5) 

^{x)=^~g4{2x-i), (6) 

<0(x),0(x-^)>= .' ^ez, (7) 

I otiierwise, 

~ , u X I 1 if i = and A; = , 

<^,^(2^x-i) > = <^ i,A;eZ, (8) 

I otherwise, 

, 7 / i, N 1 1 if i = and A; = , „ , „ 

<(j),i){2^x-i)> = { ' ^gZ,A;GZ+, (9) 

I otherwise, 

,7/1. N I 1 if i = and /c = , „ , „ , , 

< ^, 0(2^^3: - i) > = <^ ' ieZ,A;GZ_, (10) 

10 otherwise, 

where < fi{x),f2{x) >= J fi{x)f2{x) dx. We further assume that the basis 
has M dual vanishing moments: 



f x''^{x)dx = 0, k = 0,...,M-l. (11) 



It can be shown that for any scale / G Z any function / G L2(M) can be 
represented as 

oo 

fix) = J2 2V(2'a; - t)ai, + J2 2V(2'x - t)Pk„ (12) 

ieZ k=l 

where ak,i = < f, (p{2^x - i) >, (13) 

(3k,^ = <f,i^{2''x-^)>. (14) 

The fast wavelet transform, allows to quickly go in representation (12) from 
scale / to / — 1 and back {inverse fast wavelet transform) by observing that 

«z-i,i =< /, (l){2}'^x -i)>=J2 hj < /, 0(2'a; -2i- j) >= ^ hjai^2i+j, 

(15) 

A-i,i =< /, i'{2^'^x -i)>=Y, 9j < f, 0(2'a; - 2i - j) >= Y, mi,2i+j, 

j£Z jez 

(16) 



ai^i =< /, 4>{2^x - i) >= 2'-^ Y. "'-ij < 0(2'"'a; - j), 0(2'x - i) > + 

jez 

+ 2'-i Yl Pi~hj < ^(2^"'a; - j), 0(2'x - 2) > + 

oo 

+ E2'^'E/5^-i,. <^(2'a:-j),0(2'x-z)>. (17) 

k=l j€Z 

According to (10) the last term is 0. Expanding 0(2'~^a; — j) and iIj{2''^^x — j) 
by (3) and (5) and then using (7), one can show that 



1 

aii = - 



j&z jez 



In practice, if we process a discrete signal sampled with step 2^', it is typically 
assumed that in expansion (12) /3fc. = and ai^i are the discrete values of 
observed signal. 

The wavelet transform is typically used to compress or denoise signals by mov- 
ing to representation (12) with smaller / by iterative applications of (15), (16); 
setting certain f3k^i to (that will reduce data size and filter out random noise); 
and moving back to the original / by iterative application of (18). 



3 Motivation 



Let us start from considering square wavelet transform build from the D4 
Daubechies orthogonal wavelet with 2 vanishing moments, see Fig. 2. As we 
see, the (f){x)ip{y) term characterizes horizontal edges, ■?/'(a;)0(|/)-vertical edges. 
Although some authors suggested that ip{x)ip{ii) can be used to characterize 
edges in the diagonal directions, we can see that this term have different 
structure from the previous two and we can expect that the corresponding 
coefficients will decline with different speed. 

Now let us have a look at Figs. 1(a), 3(a), and 3(c). We can see that the 
diagonal squares, that are shows coefficients of il){2^x — i)%l){2^y — j) are less 
bright than others. 

Now let us consider how the value of coefficients with terms ilj{x)ip{y) and 
(f){x)tp{y), ip{x)(f){y) can be compared from the theoretical standpoint. The 
proof of the following lemma is given in Appendix: 

Lemma 1 For any wavelet basis with M dual vanishing moments and for any 
p > I there exist constants C,L such that for any <Ji,a2 > and ^1,^2 £ 11^ 
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Fig. 2. D4: the Daubechies orthogonal wavelet with 2 vanishing moments and ele- 
ments of square wavelet basis built from it. 

and for any function f{x,y) with the appropriate derivative in V'"'^: 



< f{x, y),ip{crix - 6i)4>{a2y - 62) > 

1 



< 



<C- 



M+l-l/p 



a 



i+L e2+L ^jy^ 

°"1 / "2 



f{x,y) dydx 



dx^ 



i/p 



(19) 
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(a) D4 decomposition 



(b) D4 energy distribution 




(c) CRF(13,7) decomposi- 
tion 
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(d) CRF(13,7) energy distribution 



Fig. 3. The Lena image decomposed by square wavelet transform built from the 
Daubechies orthogonal D4 filter and the JPEG2000 CRF(13,7) filter. The energy 
distribution plots in (b), (d) show the distribution of L2 norms of /3(/)(x)'0(?/), 
P'ijj{x)<f>{y) {edge detection terms) and Pip{x)ijj{y) {cross terms). 



< f{x, y), 0(crix - ei)ip{a2y - 62) > 

1 



<c 



M+l-l/p 
^2 



Q2)> 


< 


r fli+-L fl2+i 


/ °"1 / "^2 


jBl-L Ie2-L 


I <Tl 


•^2 



jM 



jig/(a;,l/)| dydx 



i/p 



(20) 



< f{x,y), ipicTix - ei)i){a2y - 62) > 
1 



< 



<c- 



CriCT2)^'^+^-^/P 



1+L 62+L 

"1 I "2 



d 



'2M 



dx^'^dy 



M 



f{x,y) dydx 



i/p 



■ (21) 



4 Non-linear approximation 



Theorem 2 For any wavelet basis with M dual vanishing moments and for 
any p > max(l, 1/M); q >1 there exists a constant C such that any function 
f{x, y) with support localized in [0, 1]^ and the mixed derivative ^jL^ m fi^, y) G 
Lp for any N > can be approximated by a function f^ such that 



\\f-fN\\L<CN^ 



-M 



j2M 



dx^dy 



M 



f{x,y) ^ log^iV, 



(22) 



where f^ has non more than CN non zero coefficients in the expansion on the 
rectangular wavelet basis. 



PROOF. Denote: 

72 



0i = (f){x - ii)0(x - 22), («i, ^2) e J, 

0i = (f){x-ii)(j){x-i2), 
^i,i = ^(2'^a; - ti)^{2'^y - ^2), {h, k) G A (^i, ^2) e J, 
^i,i = ^(2'^x-«i)V'(2'=^l/-^2), 

"i =< /,0i >, 

. .. tlx.v) 

L/r) 



V 



dx^dy 



M 



(23) 
(24) 
(25) 
(26) 
(27) 
(28) 
(29) 
(30) 
(31) 

(32) 



Because the set |0i,'?/'i,i i G X, 1 G £[ is the tensor product of the wavelet 
basis in L„( 



it forms a basis in Lg(R^) and 



/(x,i/)=5:ai0i + ^2ll'II^A,i^i,i. 



(33) 



Because f{x,y) is compactly supported, there exists a constant C such that 
for any 1 G £ 



Ar(||l||) = |{i|iGJ, A,i7^0}|<C2ll'll+C, 



(34) 



where here and below C denotes the universal constant that is maximum of 
all appropriate constants in the proof and depends only on the wavelet basis, 
p, and q. 



Denote by Iq the maximum / such that l'^M{l) < N. 
Define 



, , I M+l/p , M-l/p , 

ei = p/2'+^^'"+^r^'. (35) 



Let 



where 



/^ = E«i<^i + E2"'"EA,i^M' (36) 

~ _ ^ fo, if ||1|| > /o and A,i < eyill, .^^. 

](3\,i otherwise. 

Now let us calculate the number of non-zero coefficients N in /3i^i. From (21) 
it follows that for any fixed 1 

IIA.-lkp < CI)/2ll'll(^+^-i/^). (38) 

Let us note that if \\P\\ip < A then the number of coefficients bigger than e > 
in j3 can not exceed A^/e^. Taking into account (34), we have 

N<C^l{2^ + C)+ E ^^mTT^TM^<CN. (39) 

/=0 l=lo+l ^ ^l 



Because ip G L^o and is compactly supported, 



\\f-fN\\L,<C E ^2'q = 
l=lo+l 

n 



5 Experiments 



To numerically compare results of square and rectangular wavelet transform 
we limited ourselves to the problem of image compression. We took the stan- 
dard 512 x 512 "Mandrill", "House", "Lena", and "Barbara" greyscale images 
and compress them to reduce the number of non-zero coefficients by factors 
of 80 and 160 ^ . The results are shown in Figs 4, 5, 6, and 7. 



^ Some papers on image compression use 256 x 256 images. For the same quality, 
they report 4 times smaller compression numbers. 
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(a) The original 
"Mandrill" 512 x 512 
image. 




(b) D4 orthogonal 
wavelet, compression 
1:83, rectangular 

transform. 



(c) D4 orthogonal 
wavelet, compres- 
sion 1:83, square 
transform. 




(d) CRF(13,7) 

biorthogonal wavelet, 
compression 1:87, 
rectangular trans- 
form. 



(e) CRF(13,7) 

biorthogonal wavelet, 
compression 1:81, 
square transform. 



(f) D4 orthogonal 
wavelet, compression 
1:172, rectangular 
transform. 




(g) D4 orthogonal 
wavelet, compression 
1:167, square trans- 
form. 



(h) CRF(13,7) 

biorthogonal wavelet, 
compression 1:163, 
rectangular trans- 
form. 



(i) CRF(13,7) 

biorthogonal wavelet, 
compression 1:166, 
square transform. 



Fig. 4. The Mandrill image compressed by the rectangular and square wavelet trans- 
forms. 
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(a) The original 
"House" 512 x 512 
image. 



(b) D4 orthogonal 
wavelet, compression 
1:89, rectangular 

transform. 



(c) D4 orthogonal 
wavelet, compres- 
sion 1:86, square 
transform. 




(d) CRF(13,7) 

biorthogonal wavelet, 
compression 1:82, 
rectangular trans- 
form. 



(e) CRF(13,7) 

biorthogonal wavelet, 
compression 1:79, 
square transform. 



(f) D4 orthogonal 
wavelet, compression 
1:163, rectangular 
transform. 




(g) D4 orthogonal 
wavelet, compression 
1:166, square trans- 
form. 



(h) CRF(13,7) 

biorthogonal wavelet, 
compression 1:163, 
rectangular trans- 
form. 



(i) CRF(13,7) 

biorthogonal wavelet, 
compression 1:162, 
square transform. 



Fig. 5. The "House" image compressed by the rectangular and square wavelet trans- 
forms. 
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(a) The original 
"Lena" 512 x 512 
image. 



(b) D4 orthogonal 
wavelet, compression 
1:87, rectangular 

transform. 



(c) D4 orthogonal 
wavelet, compres- 
sion 1:86, square 
transform. 




(d) CRF(13,7) 

biorthogonal wavelet, 
compression 1:85, 
rectangular trans- 
form. 



(e) CRF(13,7) 

biorthogonal wavelet, 
compression 1:84, 
square transform. 



(f) D4 orthogonal 
wavelet, compression 
1:160, rectangular 
transform. 




(g) D4 orthogonal 
wavelet, compression 
1:166, square trans- 
form. 



(h) CRF(13,7) 

biorthogonal wavelet, 
compression 1:162, 
rectangular trans- 
form. 



(i) CRF(13,7) 

biorthogonal wavelet, 
compression 1:163, 
square transform. 



Fig. 6. The "Lena" image compressed by the rectangular and square wavelet trans- 
forms. 
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(a) The original 
"Barbara" 512 x 512 
image. 



(b) D4 orthogonal 
wavelet, compression 
1:80, rectangular 

transform. 



(c) D4 orthogonal 
wavelet, compres- 
sion 1:92, square 
transform. 




(d) CRF(13,7) 

biorthogonal wavelet, 
compression 1:88, 
rectangular trans- 
form. 



(e) CRF(13,7) 

biorthogonal wavelet, 
compression 1:83, 
square transform. 



(f) D4 orthogonal 
wavelet, compression 
1:166, rectangular 
transform. 




(g) D4 orthogonal 
wavelet, compression 
1:169, square trans- 
form. 



(h) CRF(13,7) 

biorthogonal wavelet, 
compression 1:166, 
rectangular trans- 
form. 



(i) CRF(13,7) 

biorthogonal wavelet, 
compression 1:164, 
square transform. 



Fig. 7. The "Barbara" image compressed by the rectangular and square wavelet 
transforms. 
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We believe that results clearly show that the rectangular wavelet transform 
visually outperforms the square one for all test images. Although the effect is 
more visible on the Mandrill and Barbara images and less visible on the Lena 
and House images, in all cases, the results of rectangular compression at the 
1:160 rate are visually close to the results of square compression at the 1:80 
rate. 



6 Conclusions 



As we shown, the rectangular wavelet transform has substantially better con- 
vergence rate if the approximated function has the mixed derivative of appro- 
priate order. In this case, it allows to solve the "curse of dimensionality" issue 
and gives the same degree of approximation (up to a logarithmic factor) as 
the wavelet basis in the one dimensional case. 

From the results of numerical experiments we can see that the mixed derivative 
assumption is reasonable for the standard test images and rectangular wavelet 
transform allows to improve the compression rate (measured in the number of 
non-zero coefficient) by factor of 2. 

It would be worthwhile to investigate what actual bit rate compression an in- 
dustrial grade image compression algorithm (such as JPEG2000) will produce 
if the square wavelet transform is replaced by rectangular one. 

Although further experiments in this direction are needed, we believe that it's 
worthwhile to investigate the applicabity of rectangular wavelet transform to 
video compression. Because the actual dimension of data in this case is 3, the 
solution to the " curse of dimensionality" can be particularly beneficial in this 
case. Although certain technical work to organize codec to ensure intermediate 
frame recovery is needed, the results can be quite interesting not only from 
the actual bit-rate standpoint, but also because expensive movement search 
operation is replaced by a straightforward fast wavelet decomposition. 

The rectangular wavelet transform is especially efficient to represent details 
that are either in the horizontal or vertical directions. Certain second gener- 
ation schemes (such as wavelets for the quincunx lattice) can be modified in 
to ensure the same rate of convergence for the functions with bounded mixed 
derivative as in the case of the rectangular wavelet transform. This may further 
improve the results. 
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7 Appendix: proof of Lemma 1 



Because we assumed that ip is compactly supported, tliere exists L > such 
that |a;| > L ^ iIj{x) = 0. Let us define functions "^m for m eN as follows: 



'^m{x):= / T"'~'HT)dr. 



(41) 



Because ip G L^o , ^mi.^) ^ L'-^. Moreover, because ip has M vanishing 
moments ^m is supported on [—L, L] for m < M. 

Lemma 1 is a combination of the following results: 

Lemma 3 For any function f{x) with M-th derivative 



< f{x),1p{(TX - 6) >-- 



-L / (7 



a 



M e-L 



f^\x)^Mi(rx-9)dx. (42) 



PROOF. 

< fix),^jj{ax 



>-- 

e+L 

a 

e-L 
e+L 

a 

e-L 



+ L 



f{x)ip{ax — 6) dx 



X 

e-L 



f (x) / %jj(aT — 9)dT dx 



r\x) 



e-L 



a 



M-l ~ 



i}{aT-e)dTdx. (43) 



Lemma 4 For any wavelet basis with M dual vanishing moments and for any 
p > 1 there exist constants C, L such that for any a > and 9 E M and for 
any function f with M-th derivative in L'f'^ 



< f{x),i){(jx-e) > 



<C- 



1 



M+l-l/p 



a 



e+L 

a 

e-L 



\f^^\x)'^'dx 



nl/p 



(44) 



Moreover, if the basis does not has M + 1 dual vanishing moments, then there 
exists a function f such that 



< f{x),ilj{ax - 6) > 



> 



1 1 

Q ^M+l-l/p 



e+L 



\f^^\x)\Pdx 



1/p 



(45) 



PROOF. Denote Afg := ||^m||l,- Let us note that Afq<oofoTq> 0. 
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By applying the Holder inequality to (42), we get 



<f{x),ip{ax-9) > 



< 



M+l-l/p 



a 



e+L 

a 
0-L 



\f^'^\x)\^dx 



1/p 



(46) 



To proof (45), consider /(x) = x /M\. After substituting it to (42): 



„M 



< ^,.^{cjx-e) >= -^^^^,+i(L). 



(47) 



Note that since ip does not has M + 1 vanishing moments, ^m+i{L) 7^ 0. 
Taking into the account that 



0+L 



1 dx 



i/p 



{2L/(j 



iVp 



(4J 



and putting C = max((2L)^/P/|\l/M+i(-^)|, A/i/(i_i/p), we conclude the proof. 



Corollary 5 For any wavelet basis with M dual vanishing moments and for 
any p > 1 there exist constants C, L such that for any o"i, o'2 > and 61,62 G M 



and for any function f{x,y) with the appropriate derivative in L^: 



loc . 



< fix, y),i){aix - 6*1)0(0-21/ - 62) > 

1 



< 



<C 






M+l-l/p 



01-L J 02-L I dx^ 
1 °"2 



f{x,yWdydx 



i/p 



, (49) 



< fix, y), (Pio-ix - 6i)ilj{(j2y - 62) > 

1 



< 



<C 






M+l-l/p 
^2 



0-,-L 00-L 



dy 



M 



f{x,y)\^dydx 



i/p 



■ (50) 



Lemma 6 For any wavelet basis with M dual vanishing moments and for any 
p > 1 there exist constants C,L such that for any 0^,02 > and 61,62 G M 



^2M 



loc . 



and for any function ^.^m ^yM f {x , y) E V""": 



< fix, y),^iaix - 6i)il){a2y - 62) > 



< 



<C 



(aia2)*^+l-l/p 



l+i 02+L 



0T-L 00-L 



rf2^ 



dx^dy 



M 



fix,y)fdydx 



i/p 



■ (51) 
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PROOF. Let us consider 

9{y) = J f{x, y)ij{crix -9)dx = 

1 f^ d^ 



Ih^ ■^^fi^^y)^M{(r,x-9i)dx. (52) 



^M Iot-l 



Then 

< fix, y),i){aix - 9i)^p{cr2y - 6*2) >=< g{y), ij{(T2y - 62) >= 

= ^M^j2Z ^*^(^2l/-^2)^-]^y^ ■^^f{x,y)^M{(TiX-0i)dxdy 

(53) 

Because ^^m^ m f{.x,y) and \&m are absolutely integrable, we can differentiate 
under the integral sign: 

< /(a;, y), V^(cTix - 9i)^p{(T2y - 62) >= 
~^M^joA y^ ^M{(yix-e^)^>M{(y2y-02)^^^^^j^f{x,y)dydx 

(54) 
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By applying the Holder inequality we conclude the proof. 
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